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SUMMARY 


Egon  Brunswik  first  called  attention  to  the  importance  of 
ecological  validity,  and  consequently  to  the  importance  of  cor¬ 
relations  among  cues.  Cue  intercorrelations  and  value  independence 
are  central  to  weights  in  linear  models  used  for  prediction. 

Dawes ,  Einhom  and  Wainer  have  argued  that  equal  weights  are 
often  betteT  than  least-squares  regression  weights.  Newman,  Seaver, 
Edwards  and  McClelland  have  all  shown  that  if  cue  intercorrelaticns 
are  positive  or  zero,  equal  weighting  can  lead  to  serious  errors 
of  prediction. 

Most  predictions  are  intended  as  a  basis  for  decision  making. 

The  point  of  this  paper  is  that  prediction  and  decision  require 
different  methods.  Equal  weights,  while  often  useful  for  predic¬ 
tion,  are  less  useful  for  decision  making. 

The  action  options  available  in  any  decision  problem  fall  into 
three  classes:  sure  winners,  sure  losers,  and  contenders.  Sure  winners 
and  sure  losers  are  defined  by  dominance,  accepting  sore  winners 
and  rejecting  sure  losers  is  trivial.  Good  decision,  rules  should 
discriminate  well  among  contenders. 

In  the  familiar  pick-1  decision  problem,  options  on  the  Pareto 
frontier  (i.e.  undominated  options)  almost  always  show  negative 
correlations  among  attributes.  Such  negative  correlations  make 
equal  weights  inappropriate. 

This  paper  extends  that  result  to  the  case  in  which  a  decision 
maker  must  pick  k  options  out  of  n.  In  this  case,  the  set  of  sure 
winners  is  usually  not  empty.  It  develops  general  procedures  for 
identifving  the  set  of  contenders,  given  the  options,  k,  and  n. 


This  set  is  a  generalized  Pareto  frontier,  of  which  the  traditional 
kind  is  a  special  case.  Simulations  show  that  attribute  intercorre¬ 
lations  among  contenders  are  substantially  depressed  and  typically 
negative,  even  if  the  intercorrelations  in  the  whole  set  are  positive. 
Such  negative  correlations  among  contenders  sttongly  question  the 


usefulness  of  equal  weights  for  decision  making. 
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DISCLAIMER 


I .  Introduction 


Since  Wilks  (1938)  first  published  on  the  robustness  of  equal 
weights,  many  have  argued  that  the  weighting  question  is  trivial. 

Indeed,  in  psychometrics,  differential  weighitng  of  component  scores 
of  a  test  battery  is  rare.  Formal  analytic  work  demonstrating 
the  excellent  correspondence^ between  different  sets  of  composites 
derived  from  different  weight  schemes  (Ghiselli,  1964;  Gulliksen, 

1950)  abounds  in  the  mental  tests  literature. 

This  wheel  has  been  rediscovered  many  times,  most  recently 
in  the  area  of  human  judgment  and  decision  making  and  multiple, 
linear  regression.  There  is  new  little  doubt  that  when  dimensions 
are  positively  correlated,  virtually  any  weighting  scheme  is  accept¬ 
able  (Dawes  §  Corrigan,  1974;  Einhom  8  Hogarth,  1975;  Newman,  1977; 
Wainer,  1976,  1978).  Given  agreeable  (i.e.,  non-negative)  inter- 
correlation  matrices,  it  hardly  matters  whether  the  weights  are 
obtained  subjectively,  statistically,  randomly,  or  a  priori  (i.e., 
equal  weights) ;  the  results  aTe  essentially  the  same. 

Recent  arguments  from  both  the  human  judgment  and  regression 
analysis  literatures  have  challenged  the  "non-negative  inteTCorre- 
lation  assumption."  Calling  attention  to  the  importance  ‘of  suppressor 
variables  in  multiple  regression,  Keren  and  Newman  (1978)  rejected  the 
equal  weighting  approach  as  a  general  methodology.  Negative  corre¬ 
lations  are  present,  by  definition,  in  the  case  of  suppressor 
variables;  thus,  the  one  assumption  critical  to  the  unit  weighting 
argument  is  simply  not  met  in  at  least  this  one  important  case  of  linear 
regression. 
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In  the  area  of  human  judgment  and  decision  making,  the  nan- 
negative  intercorrelation  assumption  is  even  more  tenuous.  Research 
in  multiattribute  utility  measurement  (MAUM)  by  Edwards  and  his 
associates  (Edwards,  1976;  Newman,  Seaver,  §  Edwards,  Note  1;  Seaver 
Note  2) ,  and  by  McClelland  (Note  3)  showed  that  attributes  will  be 
negatively  correlated  if  the  domain  of  available  acts  is  restricted 
to  those  on  the  Pareto  frontier.  (The  Pareto  frontier  of  any  set 
of  alternatives  consists  of  those  that  are  not  dominated.  Although 
dominance  may  be  defined  in  many  ways,  an  ordinally  dominated  act  is 
one  that  is  no  better  than  some  other  act  on  each  dimension  and  worse 
than  it  on  at  least  one  dimension.)  Of  course,  for  the  task  of  either 
describing  or  prescribing  choice  behavior,  only  those  alternatives 
on  the  Pareto  frontier  are  of  interest.  By  adding  various  dominated 
(irrelevant)  acts  one  could  generate  any  intercorrelation  matrix. 
However,  if  an  act  has  no  chance  of  being  chosen  (which  is  the  case 
for  dominated  acts),  why  consider  it  at  all? 

Working  only  with  acts  on  the  Pareto  frontier,  McClelland  (Note  3) 
Newman  et  al.,  (Note  1),  and  Seaver  (Note  2),  concluded  that  composites 
derived  from  unit  weighting  will  not  agree  satisfactorily  with  those 
obtained  from  differential  weights.  In  addition,  McClelland  (Note  3) 
Aowed  that  the  overall  value  of  the  best  composite  determined  from 
unit  weighting  may  be  substantially  less  than  that  obtained  from  the 
correct  differential  weights,  where  overall  value  is  computed  using 
the  "true"  differential  weights.  These  analytic  results  suggest  that 
the  equal  weighting  argument  is  not  applicable  to  the  raultiattribute 
decision  problem. 

The  intellectual  confusion  underlying  the  equal  weights  argument 
ir.  decision  making  arises  from  using  regression,  a  predictive  device. 
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for  decision  making.  Like  regression,  multiattribute  utility  measure¬ 
ment  (MALM)  combines  numbers  into  composites,  usually  by  rescaling  them 
and  then  taking, a  weighted  (compensatory)  combination,  usually  a 
weighted  average.  It  differs  from  regression  in  that  the  manbers 
and  weights  are  often  judged  rather  than  recovered  from  data  and 
that  the  method  of  rescaling  is  different.  It  also  differs  from 
regression  in  that  its  explicit  goal  differs  from  the  explicit 
goal  of  regression,  though  in  fact  the  two  procedures  frequently 
serve  the  same  puipose.  A  regression  coefficient,  however  calcu¬ 
lated,  is  in  fact  a  descriptive  statistic,  intended  primarily  for 
subsequent  use  as  a  tool  for  prediction.  It  seems  extremely  natural 
and  appropriate  to  base  decisions  on  the  predicted  characteristics 
of  the  options  (e.g.,  candidates  for  admission  to  graduate  school) 
being  considered.  But  doing  so  raises  a  new  set  of  problems,  quite 
unrelated  to  the  mathematics  on  which  the  prediction  mechanism  was 
based.  Some  of  these  are  examined  below. 

The  triviality  of  many  decisions.  It  is  both  a  fact  of  experi¬ 
ence  and  a  fact  of  the  mathematics  of  decision  theory  that  most  de¬ 
cisions  are  trivial.  The  point  applies  everywhere,  but  is  nicely 
illustrated  in  the  context,  standard  for  discussions  of  equal  weights, 
of  graduate  student  admissions.  Anyone  who  has  eveT  served  on  an 
admissions  committee  knows  that  same  small  fraction  of  the  admissions 
folders  require  essentially  no  discussion  before  acceptance;  the  only 
question  is  whether  the  student  will  in  fact  come,  since  one  can  be 
confident  that  there  will  be  competing  admissions  from  otheT  univer¬ 
sities.  An  even  larger  fraction  of  those  folders  require  essentially 
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no  discussion  before  rejection;  in  fact,  most  departments  make 
elements  of  the  rejection  process  automatic  by  setting  prior  stan¬ 
dards  concerning  Graduate  Record  Examination  Scores,  grade  point 
averages,  or  both.  The  result  is  that  80%  to  90%  of  the  discussion 
time  is  devoted  to  marginal  cases.  Ignoring  for  the  moment  the 
practical  problems  of  financing  students  and  of  match-up  between 
student  and  faculty  interests,  a  marginal  case  is  marginal  for  one 
or  the  other  (or  both)  of  two  reasons.  Either  various  indicators 
disagree,  in  which  case  one  is  unsure  what  to  believe,  or  all  in¬ 
dicators  are  close  to  borderline. 

Evaluating  decisions  and  evaluating  decision  rules.  Most:  of 
the  discussions  of  equal  weights,  like  most  other  discussions  of 
decisions,  evaluate  the  resulting  decisions --the  trivial  along  with 
the  difficult.  Thus,  evaluation  of  graduate  student  admissions,  if 
done  at  all,  may  be  done  by  comparing  graduate  school  grade  point 
average  (GPA)  with  undergraduate  GPA.  Obviously,  this  comparison 
will  include  all  students  admitted,  whether  the  decision  to  admit 
them  was  hard  or  easy. 

Exactly  this  kind  of  evaluation  underlies  the  argument  for  equal 
weights.  Advocates  quite  correctly  point  out  that,  by  virtually 
criterion  you  might  wish  to  specify,  the  difference  defined  over 
whole  set  of  objects  of  evaluation  between  the  merits  of  a  select 
subset  based  on  equal  weights  and  those  based  on  weights  obtained  in 
other  ways  is  often  negligible.  (Equal  weights  have  collateral  ad¬ 
vantages,  such  as  independence  of  data  and  consequent  robustness, 
but  they  are  irrelevant  to  this  discussion.) 
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There  is  a  pragmatic  virtue  in  the  process  of  evaluating  the  whole 
set  of  objects  of  choice  selected  by  a  decision  process.  But  it 
ignores  the  point  that  many  inclusions  and  exclusions  were  not  in 
fact  in  dispute;  all  decision  processes  under  consideration  would 
have  produced  them. 

That  leads  us  to  the  crucial  issue  of  this  paper.  Are  choices 
left  unaffected  by  whatever  differences  may  exist  between  decision 
rule.  .A  and  decision  rule  B  relevant  to  the  comparative  evaluation 
of  those  rules?  To  us ,  it  seems  apparent  that  the  answer  should  be 
no. 

In  other  words,  in  evaluating  the  effects  of  one  decision  rule 
(such  as  equal  weighting  combined  with  a  cut-off  an  aggre&tte  score) 
as  compared  with  another  (such  as  a  regression  coefficient  combined 
with  a  similar  cut-off) ,  one  should  ignore  those  alternatives  not 
in  actual  contention.  This  notion,  translated  into  the  technical 
language  of  ordinary  dominance  and  Pareto  frontiers,  is  the  essential 
idea  first  noted  by  Seaver,  and  later  by  Newman  and  McClelland.  But 
instances  like  that  cf  graduate  adnissiors,  3nd  man)*  others,  have  a 
slightly  more  complicated  structure  than  that  considered  by  these  re¬ 
searchers.  The  task  is,  not  to  pick  the  best  one  out  of  .n,  but  to 
pick  the  best  k  out  of  n.  The  difference  between  the  two  cases  is 
that  in  the  pick-k  case,  there  are  not  only  inevitable  losers  but 
also  .inevitable  winners.  If  Seaver' s  argument  for  excluding  the 
inevitable  losers  from  calculations  concerning  composite  correlations 
(and  inter- attribute  correlations)  is  appropriate,  an  argument  for 
exlcuding  the  inevitable  winners  would  seem  equally  appropriate. 
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for  exactly  the  same  reason- -they  are  irrelevant  elements  of  an 
ill -defined  population. 

By  now  it  should  be  apparent  that  the  central  theme  of  this 
paper  is  that,  given  a  set  of  evaluative  criteria,  the  alternatives 
being  evaluated  for  any  decision  problem  can  be  divided  into  three 
classes:  sure  winners,  sure  losers,  and  contenders.  (Of  course,  any 
of  these  sets  may  be  empty.)  First,  we  will  motivate  the  formal 
definitions  and  results  concerning  winners,  losers,  and  contenders 
with  a  simple  two-dimensional  example.  All  ideas  easily  generalize 
to  any  finite  number  of  dimensions,  substituting  hypercubes  for  squares 
hyperplanes  for  lines,  and  so  on. 

II.  A  Two-Dimensional  Example 

Since  graduate  student  admissions  policies  have  been  so  familiar 
in  the  equal  weights  literature,  we  use  them  also.  Suppose  that  a 
set  of  79  applicant  folders  is  being  considered  by  a  law  school  ad¬ 
missions  committee.  The  committee  agrees  to  base  all  decisions  on 
exactly  two  dimensions:  undergraduate  grade  point  average  (UGPA)  and 
law  school  admissions  test  (LSAT)  scores.  Ch  each  dimension  considered 
separately,  the  committee  agrees  that  higher  values  are  preferred  to 
lower.  (A  more  technical  discussion  in  an  n-dimensional  context  would 
have  to  assume  that,  for  all  possible  subsets  of  dimensions,  increase 
in  values  on  any  or  all  of  the  dimensions  cannot  decrease  aggregate 
.1  ~tiveness--a  condition  naturally  fulfilled  in  many  decision  pro- 
The  79  applicants,  plotted  in  Figure  1  (taken  fron  an 
E.w  .echnical  report,  Rubin,  Note  4),  yield  a  Pearson  product- 
moment  correlation  of  .168.  If  equal  weights  are  used  in  place  of 
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some  other  "true"  set  of  weights,  the  worst  possible  correlation  between 
composite  evaluations  is  .82  (when  the  "true"  weights  are  extreme-- 
0,  1).  Clearly,  this  is  an  example  for  which  equal  weights  would 
seem  appropriate . 

We  digress  for  a  moment  to  note  the  misleading  nature  of  that 
.168  correlation.  It  is  based  only  on  the  folders  in  the  hands  of 
the  admissions  committee- -a  strangely  defined  population.  Had 
students  who  were  prescreened  out,  or  who  chose  not  to  apply  to 
law  school,  or  who  confined  their  applications  to  more  prestigious 
or  more  convenient  schools,  been  included,  the  correlation  would 
almost  certainly  have  been  higher.  The  correlation  is  likely  to 
be  sensitive  to  many  non-population  characteristics,  such  as  the 
school's  recruiting  effort,  the  various  obstacles  that  are  placed 
in  the  way  of  completing  the  folder,  and  informal  pre-screening 
procedures.  Do  the  folders  represent  a  sample  from  a  defined  uni¬ 
verse?  In  the  absence  of  a  defined  population  and  a  suitably  random 
set  of  rules  for  sampling  from  it,  any  correlation  or  regression  co¬ 
efficient  is  only  a  statistic,  descriptive  of  the  data  in  hand.  And 
in  decision  contexts,  both  well-defined  populations  and  random 
samples  are  rare,  if  they  occur  at  all.  Consequently,  considered 
as  estimates  of  population  parameters,  such  numbers  are  doubtful — even 
if  in  actual  experience  they  replicate  from  year  to  year.  (That  simply 
means  that  non-random  processes  leading  to  inclusion  of  elements  in 
a  choice  set  have  changed  slowly,  but,  of  course,  there  can  never  be 
a  guarantee  that  they  will  remain  stable  next  time.)  We  think  this 
fact  presents  major  difficulties  in  the  application  of  the  Brunswikian 
idea  of  representativeness  to  decision  contexts.^  How  can  one  specify 
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what  is  representative,  or  what  is  being  represented?  But  that  is  not 
the  point  of  this  paper. 

Consider  first  the  case  in  which  the  admissions  conmittee  planned 
to  admit  only  one  student.  There  are  no  sure  winners.  There  are,  how¬ 
ever,  exactly  three  contenders  (A,B,C).  All  others  are  sure  losers, 
since  they  are  dominated.  The  correlation  between  UGPA  and  LSAT  among 
the  three  contenders  is  -.90>a  most  inappropriate  case  for  equal 
weights.  The  argument  we  wish  to  make  is  nothing  more  than  a  generali¬ 
zation  of  this  idea.. 

Suppose  the  admissions  committee  plans  to  admit  eight  applicants 
(roughly  10%) .  Now  there  are  some  sure  winners,  some  sure  losers,  and 
some  contenders;  our  task  is  to  identify  them.  An  applicant,  X,  is  a 
sure  winner  if  and  only  if  there  are  fewer  than  k  applicants,  y,  in  the 
total  choice  set  for  which  u  (y)^  u  (X) ,  for  any  utility  function  u. 
Thus,  if  feweT  than  k  applicants  are  better  than  X  on  either  UGPA  or 
LSAT,  X  is  a  sure  winner.  In  the  present  sample,  for  k  equal  to  eight, 
there  are  exactly  three  sure  winners  (A,  B,  D).  X  is  a  sure  loser  if 
and  only  if  there  are  at  least  k  applicants,  y,  in  the  total  choice 
set  for  which  u  (y)  >  u  (X) ,  for  any  u.  That  is,  if  k  or  more  appli¬ 
cants  dominate  X  (i.e.  better  on  both  UGPA  and  LSAT),  X.is  a  sure 
loser.  There  are  57  sure  losers.  Any  point  which  is  not  a  sure 
winner  or  a  sure  loser  is  a  general  contender.  Whether  contenders 
are  ultimately  chosen  or  not  depends  upon  the  form  and  parameter 
values  of  the  utility  function,  u.  The  19  contenders,  labelled  E 
through  V,  yield  a  correlation  of  -.78  between  UGPA  and  LSAT.  As 
in  the  case  of  k  equal  to  one,  applicant  evaluations  that  use  equal 
■weights  may  not  correlate  highly  with  those  in  which  UGPA  and  LSAT 
are  weighted  differentially. 
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By  placing  restrictions  o  n  the  form  of  the  utility  function,  u, 
the  set  of  contender  applicants  may  be  further  reduced  in  size.  For 
example ,  suppose  that  the  admissions  comnittee  decided  that  the 
evaluations  should  be  an  additive  function  of  UGPA  and  LSAT.  Graphi¬ 
cally,  any  choice  of  u  can  now  be  represented  by  a  line  with  negative 
slope,  the  magnitude  of  which  is  a  function  of  the  weights  attached 
to  UGPA  and  LSAT.  (Equal  weights  would  produce  a  line  with  slope  -1.) 

An  applicant  X  is  an  additive  sure  winner  if  every  negatively  sloped 
line,  1,  passing  through  X  yields  fewer  than  k  points  above  and  to 
the  right  of  1  (including  other  points  on  1^  -and  sure  winners).  Additive 
sure  losers  are  those  contender  applicants  for  which  evexy  line  1  through 
X  yields  k  or  more  points  above  and  to -the  right  of  1^  (again  including  other 
points  on  1^  and  sure  winners)  The  remaining  points  are  additive  general  canter 

It  is  not  possible  to  perform  the  required  calculations  for  every 
negatively  sloped  line --since  there  are  uncountably  many  of  them. 

However,  for  purposes  of  counting  applicants  above  and  to  the  right 
of  the  lines,  only  those  lines  passing  through  another  applicant 
(also  a  contender)  need  be  considered.  Thus,  points  must  be  counted 
for  only  18  different  lines  for  each  of  the  19  contenders  (342  total). 

Of  the  19  general  contenders,  8  are  additive  sure  losers,  while  the  re¬ 
maining  11  are  additive  contenders.  Although  no  additional  sure 
winners  were  obtained,  this  is  not  necessarily  the  case  for  all 
problems.  Among  the  additive  contender  applicants,  UGPA  and  LSAT 
are  correlated  -.79.  Again,  this  case  is  not  well  suited  for  equal 
weighting. 

Initially,  the  decision  problem  was  to  select  8  entering  students 
for  a  law  school  from  a  set  of  79  applicants.  Basing  the  selections 
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upon  (monotonic)  functions  of  UGPA  and  LSAT  reduced  the  problem 
to  one  with  only  19  contenders  for  5  slots.  Basing  applicant 
evaluations  upon  a  weighted  average  of  UGPA  and  LSAT  reduced  the 
set  of  contenders  to  only  11.  Thus,  the  fates  of  at  least  761 
of  the  applicants  were  determined  independently  of  weights;  over 
a  third  of  the  admissions  made  did  not  depend  on  weights.  However, 
among  the  11  applicants  to  whjch  the  decision  rule  was  relevant, 
the  ordering  (and  subsequent  selection)  was  highly  sensitive  to 
the  weights. 

III.  A  Simulation 

Having  motivated  the  central  ideas  necessary  for  an  analysis  of 
the  pick  k  out  of  n  problem,  the  remainder  of  the  paper  is  devoted 
to  a  Monte  Carlo,  investigation.  Using  the  pivot  procedure  of  matrix 
factoring,  160  m- tuples  were  generated  from  a  multivariate  normal 
distribution  (see  Newman,  Seaver,  £  Edwards, Note  1).  A  breakdown 
of  the  Monte  Carlo  design  is  given  in  Table  1.  The  covariance  struc¬ 
ture  of  the  multivariate  noimal  distributions  of  alternatives  main¬ 
tained  equal  expected  correlations  among  all  pairs  of  attributes 
(eitner  -.2.  0.0.  0.5.  or  0.9).  The  pioportion  of  alternatives  to 
be  chosen,  k,  was  varied  for  values  less  than  50%,  since  all  dependent 
variables  were  expected  to  be  symmetric  about  k  *  501.  (Preliminary 
runs  verified  that  this  was  indeed  true.)  Data  for  all  ten  of  the 
within- sample  factor  cells  were  obtained  from  each  of  the  160  gene¬ 
rated  samples.  For  the  various  combinations  of  factor  levels,  each 
of  five  sairples  were  partioned  into  three  groups:  (additive)  sure 
winners,  (additive)  sure  losers,  and  (additive)  general  contenders. 

The  number  of  alternatives  in  each  of  these  three  sets,  as  well  as 


TABLE  1 


Monte  Carlo  Design 


Factor  No.  of  Levels 

 Levels 


No.  of  points  (density) 

2 

50,  100  points 

No.  of  dimensions 

2,  3,  4,  5  dimensions 

Ty  in  initial  interattribute 
correlation  matrix 

4 

-0.2,  0.0,  0.5, 

0.9 

Total  No.  of  Between  Cells 

32 

Proportion  of  points  chosen 

5 

2,  10,  20,  30, 

40  1 

Utility  function  restrictions 

2 

None,  Additive 

Total  No.  of  Cells 

320 

Repetitions 

5 

Total  No.  of  Cases 

1600 

- 

the  average  inter -attribute  correlation  among  the  contenders,  was 
computed. 

The  problem  of  obtaining  the  additive  partitioning  is  solvable  as  a 
non-linear  programming  problem  (non-linear  and  non -continuous  objective 
function,  linear  constraints) .  The  rei  esentaticn  of  weight  vectors 
as  lines  in  two  dimensions  was  generalized  to  hyperplanes  in  m  dimen¬ 
sions.  Thus,  for  each  of  the^c  general  contenders,  (^  ’  ^)  different 
weight  vectors  must  be  solved  for  and  considered.  Thus,  for  each 

r  .  1 

repetition,  c  •  (^  _  ^)  matrices  of  size  m  x  m  must  be  inverted.  Table 
2  displays  this  msriber  for  values  of  m  ■  2,  4,  8,  and  c  *  10,  50,  100. 
The  computing  time  and  costs  for  arithmetic  of  this  volume  are  pheno¬ 
menal,  if  not  absolutely  prohibitive.  Thus,  another  solution  for  the 
additive  case  was  sought. 

Fishbum  (1965)  proved  a  dominance  theorem  based  an  Abel's  well 

known  simulation  identity.  The  theorem  provides  a  method  for  eliminating 

strategies  (from  the  ordinary  outcome  by  event  pay-off  matrix)  based 

on  rank  order  information  about  the  event  probabilities.  Of  course, 

since  the  mathematical  structure  of  the  MALM  problem  is  identical  to 

that  of  ordinary  decision  theory,  the  same  theorem  applies  to  weights. 

Thus,  for  any  given  rank  ordering  of  the  weights,  one  can  partition 

2  * 

the  alternatives  into  winners,  losers,  and  contenders.  If  one  per¬ 
forms  these  calculations  for  every  possible  rank  ordering,  then  the 
intersection  of  all  of  the  sets  of  sure  winners  will  be  the  set  of 
additive  sure  winners.  The  intersection  of  all  sets  of  sure  losers 
will  be  the  set  of  additive  sure  losers.  Of  course,  the  remaining 
alternatives  are  additive  general  contenders. 
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TABLE  2 

Number  of  jn  x  m  Matrices  to  be  Inverted 
For  m  Dimensions  and  c  General  Contenders 


of  Contenders  (c) 

No.  of  Dimensions  (m) 

_2 

A_ 

_8_ 

10 

90 

840 

360 

SO 

2450 

9.2  x  105 

4.3  x  109 

100 

9900 

1.6  x  107 

7.3  x  1013 
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Since  the  results  are  essentially  identical  for  the  two  levels 
of  total  number  of  points  (50  and  100) ,  all  of  the  results  will  be 
reported  for  100  points  only.  The  proportion  of  sure  winners  is 
displayed  in  Figures  2  and  3.  Each  data  point  represents  five 
repetitions.  The  maximum  possible  value  for  this  proportion  is 
the  k%,  represented  by  the  long -short  dashed  line.  The  main  effects 
here  are  for  initial  attribute,  intercorrelation  and  number  of  dimen¬ 
sions.  As  the  attributes  become  more  highly  correlated,  the  percen¬ 
tage  of  sure  winners  increases  dramatically.  This  result  is  atten¬ 
uated  for  higher  numbers  of  dimensions.  The  increase  in  sure  winners 
caused  by  the  additive  utility  restriction  is  quite  modest.  Also, 
the  proportion  of  sure  winners  seems  to  increase  roughly  linearly  with 
the  proportion  to  be  chosen  (k). 

Proportions  of  general  contenders  are  displayed  in  Figures  4  and 
5.  These  results  are  complementary  with  those  fbT  sure  winners: 
higher  numbers  of  dimensions  mean  more  contenders,  and  higher  initial 
correlations  among  attributes  signal  fewer  contenders.  We  suspect 
that  the  contenders  peak  at  k  =  501  (roughly  the  same  as  k  *  40%) 
because  of  the  multivariate  normal  distributions  used.  Had  the  dis¬ 
tributions  of  alternatives  been  skewed,  the  proportion  of  contenders 
would  not  have  been  symmetric  about  k  ■  501 ,  and  the  k  «  50%  peak 
would  not  have  resulted.  There  seems  to  be  a  somewhat  larger  effect 
for  the  additive  restriction:  if  one  can  assume  that  u  is  additive, 
then  substantially  fewer  alternatives  need  be  considered  at  all. 

The  results  for  the  average  intercorTelation  among  pairs  of 
attributes  for  general  contenders  are  presented  in  Figures  6  and  7. 


20  30  40 


PROPORTIONS  OF  GENERAL  CONTENDERS -Unrestricted 


PAGE  18 

0} 

O  id  ro  OJ 

<i  q  9  <? 
\  \  '  » 

<  t!  ^ 
\  \  \ 

nA  ' 

CL  tX 


o 

ro  ^ 

O  ^ 
OJ 


Sd3GN3±N0D  JO  NOIldOdOdd 


r* 


■  Am#  "*M.  5  -  •  • •  V  -•  .*  -•  .•  ■“  »' 


.  •.  ,  *,  s  % 

v. 


PROPORTIONS  OF  GENERAL  CONTENDERS -Additive 


30  40  2  10  20  30  40 


AVERAGE  ATTRIBUTE  INTERCORRELATIONS  AMONG 
GENERAL  CONTENDERS  —  Unrestricted  u 


10  20  30  40 


PAGE  22 


The  most  profound  effect,  here  is  for  number  of  dimensions.  The 
average  correlation  rises  rapidly  as  the  number  of  dimensions 
increases.  The  average  attribute  intercorrelations  rise  mono- 
tonically  as  a  function  of  the  initial  correlation  among  attri¬ 
butes.  Thus,  high  initial  correlations  produce  both  fewer  con¬ 
tenders  and  higher  attribute  correlations  among  the  contenders. 

The  effects  of  k%  and  the  additive  form  restriction  are  not  very 
great.  In  general,  the' additive  general  contenders  produce  more 
negative  attribute  intercorrelations  than  do  the  nan-u- restricted 
general  contenders.  The  attribute  intercorrelations  also  increase 
as  a  function  of  k%.  It  is  noteworthy  that  the  average  attribute, 
intercorrelations  for  the  pick  21,  r^  *  -.20  case  (fax  left  panel) 
are  nearly  equal  to  the  smallest  values  possible.  For  2,  3,  4,  and 
5,  dimensions,  the  least  possible  correlations  are  -1,  -.5,  -.33, 
and  -.25,  compared  to  the  values  -.85,  -.40,  -26,  and  -.21  obtained. 
The  inter-attribute  correlations  among  contenders  for  the  k  -  2t , 
r^  ■  0  case  are  in  good  agreement  with  the  ones  McClelland  published 
in  the  pick-one  case  using  hypothetical  "typical"  Pareto  frontiers. 

IV.  Conclusions 

Our  simulation  produced  two  main  results.  The  first  -is  the 
triviality  of  many  decisions  in  a  multiattribute  decision  context 
where  the  problem  is  to  accept  k  out  of  n  candidates.  We  showed 
that  two  conditions  guarantee  a  large  proportion  of  sure  winners  and 
sure  losers:  if  attributes  are  positively  correlated  and  if  the 
number  of  attributes  is  relatively  small.  Correspondingly,  the  pro¬ 
portion  of  contenders  decreases  with  increasing  attribute  inter¬ 
correlations  and  decreasing  number  of  attributes. 


We  had  speculated  that  the  reduction  of  contenders  would  severely 
depress  intercorrelations  between  attributes.  Indeed,  this  is  the 
second  result  o,f  our  simulation.  In  the  two  attribute  case  the 
attribute  intercorrelation  becomes  highly  negative,  independent  of 
the  initial  correlation  in  the  whole  set,  independent  of  the  accept¬ 
ance  ratio  (k/n),  and  independent  of  the  restrictions  an  the  utility 
function.  This  is,  of  course^ a  generalization  of  McClelland's  findings. 
The  depression  effect  is  maintained  for  higher  number  of  attributes, 
although  the  average  correlation  among  attributes  is  less  highly 
negative,  and  in  same  cases  (i.e.  when  the  original  correlations 
are  .9)  even  slightly  positive. 

The  increase  in  the  average  correlation  for  higher  numbers  of 
attributes  was  expected.  It  is  well  known  that  the  constraints  imposed 
on  a  correlation  matrix  create  a  lower  bound  on  the  average  intercor¬ 
relation,  which  tends  to  go  to  zero  for  large  n.  On  the  surface  this 
seems  to  contradict  our  argument  for  the  prevalence  of  negative 
correlations  among  attributes.  However,  near  zero  average  correlations 
can  be  obtained  by  high  correlations  with  opposite  signs.  While  our 
simulation  does  not  provide  any  direct  evidence  of  such  a  correla¬ 
tional  structure,  the  real  world  often  does.  Consider,  for  example, 
the  attributes  characterizing  cars,  which  are  usually  grouped  into 
"cost"  and  "quality."  The  market  place  and  scarce  resources  produce 
negative  correlations  between  cost  and  quality.  Lower  level  attributes 
are  probably  positively  correlated  within  each  of  these  two  general 
dimensions  and  negatively  across,  thus  producing  exactly  the  kind  of 
opposite  sign  correlation  matrix  which  can  cause  weights  to  be  sensi¬ 
tive.  We  believe  that  such  correlational  structures  are  relatively 


ccmtnon  among  contender  sets.  The  underlying  reason  is  the  hierar- 
ical  organization  of  values  in  which  the  highest  level  objectives 
are  usually  in  conflict. 

The  arguments  favoring  equal  weights  rely  on  zero  or  positive 
inter-attribute  correlations.  Our  results  suggest  that  such  positive 
correlation  structures  may  be  more  the  exception  than  the  rule  in 
raultiattribute  decisions.  We  conclude  therefore  that  the  equal 
weights  argument  cannot  easily  be  generalized  from  its  original 
domain  (prediction,  with  usually  high  positive  intercorrelations 
among  predictors)  to  multiattribute  decision  making  (with  often 
severly  depressed  or  negative  attribute  intercorrelatians) . 

From  the  depressed  and  negative  inter-attribute  correlations 
we  can  further  conclude  that  the  correlations  among  composites  between 
equal  and  non-equal  weighting  schemes  will  be  much  lower  when  re¬ 
stricting  the  set  of  contenders  than  when  applied  to  the  whole  set. 

Thus,  in  generalization  of  previous  results,  we  believe  that  equal 
weights  will  often  do  poorly  in  multiattribute  decision  making  when 
evaluated  on  the  basis  of  correlations  among  composites. 

Vo  ualce  a  final  evaluation  of  equal  weights  as  an  approximation  rule  in 
MAUM,  we  would  have  to  go  one  step  further,  and  analyse  losses  in  - 
utility.  It  was  exactly  this  spirit  of  losses  in  utility  which  led 
us  to  examine  contenders  in  the  first  place,  since  sure  winners  and 
sure  losers  can  never  lead  to  a  loss  in  utility.  McClelland  analyzed 
a  special  foim  of  utility  loss  in  the  two  attribute  case  and  concluded 
that  equal  weights  would  do  poorly  if  the  Pareto  frontiers  close 
to  a  straight  line.  V.  Winterfeldt  and  Edwards  (Note  S)  studied 
utility  losses  at  the  Pareto  frontier  for  a  higher  msnber  of  dimensions 
and  argued  that,  in  general,  differential  weighting  will  not  lead  to 
large  utility  losses.  No  one  has  examined  utility  losses  for  the  k 
of  n  problem  yet . 
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FOOTNOTES 

^Egon  Brunswik  (Hammond,  1966)  first  called  attention  to  the 
importance  of  ecological  validity,  and  consequently  to  the  impor¬ 
tance  of  correlations  among  cues.  He  correctly  pointed  out  that 
orthogonal  sets  of  cues  exist  only  in  the  psychological  laboratory. 

He  also  argued  that  the  set  of  correlations  among  cues  should  in 
some  sense  represent  "the  causal  texture  of  the  environment." 

He  interpreted  this  is  to  mean  that  they  should  be  sampled  in  some 
sense  representatively  from  the  set  of  environmental  inter-cue 
correlations.  While  his  methods  were  informal,  some  case  can  be 
made  that  he  did  in  some  sense  attempt  to  accomplish  this,  although 
of  course,  he  could  not  define  the  universe  of  inter-cue  correlations 
and  so  could  not  .use  formal  sampling  methods. 

For  technical  details  of  the  procedure,  see  Fishbum  (1965) 
and  Barron  (1973) . 
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Egon‘  Brunswik  first  called  attention  to  the  importance  of  ecological  validit] 
and  consequently  to  the  importance  of  correlations  among  cues.  Cue  intercor¬ 
relations  and  value  independence  are  central  to  weights  in  linear  models  used 
for  prediction.  Dawes  and  Corrigan,  Einhom  and  Hogarth,  and  Wainer  have  argt 
that  equal  weights  are  usually  better  than  least-squares  regression  weights. 
Newman,  Seaver,  Edwards  and  McClelland  have  all  shown  that  if  cue  intercorre- 
lations  are  positive  ot  zero,  equal  weighting  works  very  well,  but  if  cue 
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intercorrelations  are  negative,  equal  weighting  can  lead  to  serious  errors 
of  prediction. 

Most  predictions  are  intended  as  a  basis  for  decision  making.  The  point 
of  this  paper  is  that  prediction  and  decision  require  different  methods. 

Equal  weights,  while  often  useful  for  prediction,  are  less  useful  "for  decisior 
making . 

The  action  options  available  in  any  decision  problem  fall  into  three 
classes:  sure  winners,  sure  losers,  and  contenders.  Sure  winners  and  sure 
losers  are  defined  by  dominance;  accepting  sure  winners  and  rejecting  sure 
losers  is  trivial.  Good  decision  rules  should  discriminate  well  among  con¬ 
tenders  .  \ 

In  the  familiar  pick-1  decision  problem,  options  on  the  Pareto  frontier 
(i.e.  undominated  options)  almost  always  show  negative  correlations  among 
attributes.-  Such  negative  correlations  make  equal  weights  inappropriate. 

This  paper  extends  that  result  to  the  case  in  which  a  decision  maker 
must  pick  k  options  out  of  n.  In  this  case,  the  set  of  sure  winners  is 
usually  not  empty.  It  develops  general  procedures  for  identifying  the  set 
of  contenders,  given  the  options,  k,  and  n.  This  set  is  a  generalized 
Pareto  frontier,  of  ttfiich  the  traditional  kind  is  a  special  case.  Simulation 
show  that  attribute  intercorrelations  among  contenders  are  substantially 
depressed  and  typically  negative,  even  if  the  intercorrelations  in  the  whole 
set  are  positive.  Such  negative  correlations  among  contenders  strongly 
question  the  usefulness  of  equal  weights  for  decision  making. 
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